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Introductory  Remarks 

Although  this  is  being  presented  as  a  Final  Report,  it  is  important  to  emphasise  that  this  is  a 
four  year  on-going  research  collaboration  for  which  this  report  details  the  first  years 
achievements  under  grant  number  12-1-4063. 

Also  notable  is  the  intensely  collaborative  nature  of  what  we  are  attempting  and  after  the  first 
year  the  team  has  worked  well  together  on  several  of  the  sub  projects  and  made  plans  for 
directions  and  work  in  the  remaining  3  years.  As  a  result,  the  publications  and  projects 
completed,  as  detailed  below,  are  not  just  the  output  of  the  Marshall  laboratory  and  are 
contributed  to  by  Cronin,  Roberts  and  Gruev  as  well  as  our  respective  postgrads  and  PhD 
students. 

The  productivity  of  our  combined  efforts  are  also  greatly  enhanced  by  being  able  to  build  on 
previously  funded  AOARD  and  AFOSR  grants  and  contracts. 

The  original  objectives  of  this  collaborative  four  year  project  are  detailed  below  with  details 
of  the  outputs  this  year.  Those  components  that  are  expected  to  be  covered  /  achieved  in  the 
2014  year  are  coloured  red. 


Year  1  achievements  summary 


•Behavioural  tests  of  colour  and  polarisation  show  coarse  discrimination  in 
stomatopods.  New  form  of  visual  processing  implicated? 

•First  real  time  underwater  linear  polarisation  camera  built  and  tested  in  field. 

•Polarisation  properties  of  reef  animals,  reflections,  camouflage  and  light  field 
baseline  data  begun. 

•Photonic  properties  of  silvery  guanine  crystals  in  fish  skin  determined  and 
published  in  Nature  journals.  Application  in  LED  /  light  guides? 

•First  fluorescent  staining  of  stomatopod  eye  and  brain  neuroarchitecture. 

•Ultrastructure  of  polarisation  signal  mechanism  in  stomatopods  modelled 


(1)  To  learn  the  design  foundations  stomatopods  use  for  optimized,  rapid  processing  of 
multi-channel  information.  We  will  examine  the  limits  of  resolution  polarization  vision  in 
stomatopods,  using  new  methods. 

Question  1.  What  is  the  sensitivity  of  polarization  vision  systems?  Using  new  behavioural 
techniques,  stomatopod  and  other  invertebrate  polarisation  sensitivities  will  be  described. 

In  2012-13  we  determined  that  stomatopods  have  surprisingly  coarse  spectral  and 
polarisation  resolution,  using  this  new  behavioural  technique.  Conversely  polarisation 
resolution  in  cuttlefish  and  fiddler  crabs  is  fine.  As  with  colour  processing  (Fig.  2)  this 
indicates  a  new  form  of  information  processing. 

In  2014  we  will  develop  models  and  circuit-diagrams  that  account  for  this  difference  between 
stomatopods  and  other  animals,  test  other  stomatopod  species  and  other  non-stomatopod 
species  for  polarisation  resolution. 


Animals  react  to  a  looming  stimulus 
on  screen  but  only  if  they  possess 
polarisation  vision.  Humans  can  not 
see  these  stimuli  (see  video?). 


Figure  1  Polarisation  test  paradigm  A-C  Left,  show  setup  with  cuttlefish.  The  looming  black  circle  can  only 
be  seen  by  animals  with  polarisation  vision  and  their  reaction  is  recorded.  Cuttlefish  blanch  or  change  body 
colour  as  shown  in  B  -  top  half  before,  bottom  half  after  stimulus.  This  can  be  quantified  using  pixel  brightness 
and  is  sensitive  down  to  one  degree  of  polarisation  difference.  Stomatopods  (Right)  also  react  to  similar  looming 
stonuli  by  retreating  or  showing  eye  movements.  However,  the  reaction  threshold  is  far  coarser  than  expected 
and  is  closer  to  15  degrees. 


Question  2.  How  are  stomatopod-type  line  scan  systems  optimized  for  multi-channel 
acquisition  and  information  processing? 


Using  tests  for  colour  vision  (and  with  manuscript  currently  submitted  to  Science),  we  have 
shown  conclusively  that  stomatopods  process  information  in  parallel  and  without  comparison 
between  input  channels.  This  is  a  completely  new  way  to  analyse  information  and  means  the 
stomatopod  system  may  operate  to  examine  surfaces  as  a  full  spectrum  ellipsometer  might 
build  information  to  populate  a  matrix. 
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Figure  2  Proposed  processing  mechanism,  a)  Idealised  spectral  reflectance  from  stomatopod  body  parts  b) 
Spectral  sensitivities  throughout  the  spectrum  divided  into  separate  bins,  c)  and  d)  Excitation  patterns  of  each 
spectral  sensitivity  when  looking  at  the  blue  c)  and  the  red  d)  reflectance  spectra,  e)  Polarisation  system  in 
stomatopods  with,  respectively,  linear  (90°  angles),  elliptical  (45°  angles)  and  circular  (circles)  polarisation 
sensitivities  binned  in  the  same  way  as  the  spectral  sensitivities. 


Question  3. 


Do  animals  with  polarization  sensitivity  see  further  in  scattering  media?  This  is  one  of  the 
experiments  to  be  conducted  in  collaboration  with  the  Roberts  lab  in  June  this  year. 

Preliminary  results  from  these  experiments,  conducted  in  the  field  on  The  Great  Barrier  Reef, 
over  two  field  trips  suggest  that  PS  can  aid  visual  distance  in  turbid  media  in  fish. 

In  2013  this  successful  experimental  protocol  was  transferred  to  stomatopods  and  will 
continue  in  2014  with  the  addition  of  a  new  PhD  student  to  the  Roberts  laboratory. 


Improved  vision  through  haze  using 
polarisation  vision? 


Scheduler  algorithm 
Doubles  underwater 
visibility  range ! ! 


j — best  polarization  state  — |  | —  recovered  image 


Figure  6.  A  scene  26m  deep  under  the  water  surface.  The 
images  were  taken  using  horizontal  and  vertical  polarizer  ori- 
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Stomatopod  rotational  eye  movement  to  optimise  2D  system 
2D  colour  systems  also  use  inverted  approach  to  recover  image? 
Does  this  algorithm  exist  in  nervous  system? 


Figure  3  Greater  visual  penetration  through  de-hazing.  In  2003  Yoav  Scheduler  and  colleagues  combined 
machine  vision  principles  and  some  inspiration  from  stomatopod  vision  to  hypothesise  that  animals  or  sensor 
systems  with  only  two  channels  of  optimised  linear  polarisation  could  remove  intervening  polarisation  scatter. 
The  result  is  an  improvement  of  underwater  visual  range  of  up  to  50%.  We  are  now  testing  this  prediction  with 
both  stomatopod  and  fish  models. 


(2)  To  discover  how  information  is  processed  in  stomatopod  visual  systems.  At  higher 
levels,  we  will  examine  neural  interconnectivity  and  processing  principles  at  various  stages  of 
analysis  in  the  visual  system  both  anatomically  and  with  electrophysiological  approaches. 

Question  1.  How  do  the  photoreceptor  cells  of  mantis  shrimps  produce  responses  that  track 
extremely  transient  changes  in  stimulus  intensity? 

Extracellular  recordings  of  stomatopod  visual  responses  are  now  complete  and  will  tell  us 
that  stomatopods  have  a  relatively  rapid  response  time  compared  to  other  animals.  Given  the 
rapid  behaviours  they  engage  in,  including  ballistic  strikes  and  fighting  behaviour,  this  is 
expected. 

In  2014  we  will  attempt  to  calculate  if  this  visual  rapidity  is  a  result  of  the  coarse  nature  or 
low  bit-rate  of  the  system. 
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Figure  3  Spectral  discrimination  curves  (Ak/k).  Spectral  discrimination  curve  from  behavioural  testing  of  H. 
trispinosa  in  thick  black  line,  modelled  spectral  discrimination  curve  in  thick  dashed  line.  Figure  modified  from 
previous  work  on  different  animals  as  comparison.  This  indicates  a  low  bit-rate  for  the  stomatopod  system. 


Question  2.  How  are  the  neural  circuits  constructed  to  provide  the  information  processing? 
This  is  a  large  question  but  will  be  started  in  2013  along  with  the  new  PhD  studentship  and 
PostDoc  for  the  project. 

PhD  student  Hanne  Thoen  and  New  PhD  student  to  the  project  Rachel  Templin  have  begun 
high  resolution  transmission  electrom  microscopy  and  fluorescent  staining  of  the  stomatopod 
nervous  system  to  determine  information  pathways  arising  from  the  various  visual  modalities. 
This  will  continue  through  2013. 

Question  3.  What  messages  reach  the  stomatopod  brain  in  a  way  that  allows  rapid  decision 
making? 


Neuroanatomy  and  Computational  models  allow  estimates  of 
polarisation  contrast  signals 


Polarisation  information 


Simple  neural  wiring  models  can 
Be  used  to  predict  how  polarisation 
Information  is  encoded  by  the  brain 


Figure  4  Neural  connectivity  and  modelled  polarisation  discrimination  system.  Fluorescent  labelling  of 
stomatopod  brain  structures  are  revealing  the  information  flow  to  the  central  processor.  This  enables  models  of 
the  discrimination  system  to  be  developed  where  the  polarisation  difference  between  object  and  background  is 
estimated. 


(3)  To  discover  the  natural  complexity  of  visual  scenes  the  stoma topod  imaging  system 
has  evolved  to  see.  It  is  fundamental  that  we  understand  the  visual  information  stomatopods 
see. 

Question,  development  1.  Building  a  real-time  Schechner- algorithm  camera  for  visual 
enhancement  underwater.  Development  of  real-time  polarisation  cameras  with  collaborator 
Gruev  is  underway  and  we  expect  to  trial  V2  systems  in  June  on  The  Great  Barrier  Reef. 

In  2012  and  2013  V2  cameras  were  tested  with  great  success  on  the  GBR.  This  camera  is  in 
final  stages  of  housing  manufacture  and  will  be  ‘released  to  the  biologisys’  in  October  this 
year. 

2014,  Circular  polarising  cameras  will  be  developed  and  tested. 

Question,  development  2.  Camouflage  and  communication  underwater  using  polarized  light. 
Imaging  of  animals  in  their  natural  habitat  is  continuing  and  should  be  almost  completed  in 
this  year. 


Silvery  camouflage  was  examined  in  2012  with  a  Nature  publication  resulting  from  this 
work.  In  2013,  further  camouflage  and  communication  in  polarisation  space  were  examined 
in  both  underwater  and  terrestrial  habitats.  Field  observations  support  the  idea  that  silvery 
fish  do  not  camouflage  well  against  polarising  backgrounds  and  are  easy  prey  to  eg 
cephalopod  predators. 


|  output  15  1  j.n-jjJ 


First  real-time  underwater  polarisation  camera 


Intensity  channel 


Percent  polarisation 


Angle  of  polarisation 


Figure  5  Camera  design  and  real-time  video  frame  results.  Silvery  fish  are  relatively  well  camouflaged  in 
intensity  and  angle  but  are  conspicuous  in  degree. 


Question,  development  3.  Combined  linear  and  circular  polarization  cameras.  This  is 
underway  with  Gruev  lab. 

A  series  of  bio-inspired  polarisation  cameras  with  different  functions  and  application  is 
planned  for  the  remaining  years  of  the  project 

Question,  development  4.  Enhanced  optical  storage  and  information  efficiency? 

This  is  a  year  3,4  question,  once  we  know  more  about  the  pathways  to  the  brain  and  the 
information  content. 

Question,  development  5.  A  new  generation  of  color-polarisation  cameras  with  stacked  rather 
than  adjacent  color  channels.  Another  ‘technology’  already  employed  by  stomatopods. 

A  year  3.4  question  also. 


(4)  To  investigate  the  co-evolution  of  visual  systems,  visual  signals  and  camouflage  in 

the  natural  environment.  We  will  proceed  from  the  above  discoveries  to  relate  how 
stomatopods  see  and  what  they  see  to  our  observations  that  species  vary  in  the  types  of 
signals  they  produce. 

Question  1.  Are  the  polarization  signals  from  different  stomatopod  species  matched  to 
species -specific  elliptical  polarization  vision  capability?  New  anatomical  studies  will  be 
initiated  this  year  to  examine  this  question. 


Polarisation  cameras  detect  polansmg 
Signals  on  cuttlefish  and  stomatopods 
They  encode  this  in  false  colour 


We  assume  these  patterns  are  used  for 
Communication  but  do  not  know  any 
Details  of  this  secret  language  yet. 


Figure  6  Polarisation  signals.  Stomatopods  and  cephalopods  both  produce  polarising  signals.  The  photonic 
method  of  production  of  these  signals  is  under  investigation  with  new  results  expected  in  2014. 


Question  2.  As  well  as  between  species  R8  differences  (and  therefore  elliptical  information 
tuning),  very  recent  results  suggest  within- species  differences  in  some  species  (Figs.  8&9).  Is 
this  the  outcome  of  an  ‘arms  race’  in  which  there  are  generalist  species,  able  to  interpret  the 
messages  of  prey,  break  prey  camouflage,  and  ‘read  the  mail’  of  the  unsuspecting 
messengers? 

In  2014  new  ultra-high  resolution  electron  microscopy  will  be  used  to  optically  model  the 
retardation  properties  of  different  R8  cells.  Use  of  this  system  was  delayed  in  2013. 


Students  and  Marshall  lab.  personnel 

The  personnel  involved  in  and  supported  by  this  project  are  summarized  below: 


Justin  Marshall  PI 


Martin  How 
Yakir  Gagnon 
Yi-Hsin  Li 
Shelby  Temple 


PostDoc,  salary  supported 
PostDoc,  salary  supported 
PostDoc,  research  supported  50% 
PostDoc,  research  supported 


Hanne  Thoen 
Wen- Sung  Chung 
Rachel  Templin 


PhD,  salary  supported  50%  (started  April  2011) 
PhD,  research  supported 
PhD,  research  supported 


Where  possible,  we  have  sought  to  build  the  capacity  of  the  grant  through  awards  for 
students,  travel  and  personnel  costs.  Example  from  the  above  list  include  PhD  awards  from 
Norway,  Australia  and  Taiwan. 


Publications  in  2012-13 


SE  Temple,  V  Pignatelli,  T  Cook,  MJ  How,  T-H  Chiou,  NW  Roberts  and  NJ  Marshall  2012. 
High  Resolution  polarisation  vision  in  a  cuttlefish.  Curr.  Biol.  22:R121. 


NJ  Marshall,  KL  Cheney.  2012  Vision  and  body  colouration  in  marine  invertebrates.  In:  The 
Visual  Neurosciences  eds  Chalupa  et  al.  MIT. 

MJ.  How,  Pignatelli,  V.,  Temple,  S.E.,  Marshall,  NJ.  and  Hemmi,  J.M.  2012  High  e-vector 
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Experimental  Biology.  215:2128-34. 

N.W.  Roberts,  N.J.  Marshall,  and  T.W.  Cronin.  2012.  High  levels  of  reflectivity  and  pointillist 
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cuttlefish  camouflage  using  in  situ  spectrometry.  Biological  Journal  of  the  Linnean 
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Princeton  University  Press. 
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Conference  on  Invertebrate  Vision. 
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Research  Team  Structure 


The  research  program  brings  together  laboratories  in  three  different  countries,  Cronin  (USA), 
Marshall  (Australia),  Roberts  (Bristol,  UK)  and  Viktor  Gruev  (Missouri,  USA). 


Almost  all  of  the  questions  to  be  examined  above  are  collaborative  with  this  team  and  I  have 
indicated  this  where  possible  so  far.  To  summarise  this,  we  expect  milestones  to  be  achieved 
in  the  following  approximate  proportions: 

1)  Efficiency  and  optimization  in  behaviour  and  task-solving. 

Roberts  34% 

Marshall  33% 

Cronin  33% 

2)  Information  processing,  from  molecular  to  behavioural  levels. 

Cronin  35% 

Marshall  35% 

Roberts  20% 

Gruev  10% 

3)  Testing  biological  principles  through  novel  bio-inspired  camera  design. 

Gruev  50% 

Roberts  30% 

Marshall  10% 

Cronin  10% 

4)  The  co-evolution  of  seeing  and  signals  for  optimal  efficacy. 

Marshall  34% 

Cronin  33% 

Roberts  33% 


The  diagram  on  the  following  page  indicates  the  relative  interactions  of  the  four  groups: 


